command expansions #3393

QiBaobin · 2022-08-11T13:28:25Z

#3134

QiBaobin · 2022-08-11T13:28:55Z

only support filename val for now:
bc %val{filename}

support simple sh command like:

open %sh{fd\ 16_theme}

sbromberger · 2022-08-12T00:52:20Z

Just curious: does this quote the filename so that if you pass it to sh it doesn't fall victim to the IFS boundary?

QiBaobin · 2022-08-12T01:26:52Z

Just curious: does this quote the filename so that if you pass it to sh it doesn't fall victim to the IFS boundary?

I think we can use sh echo "%val{filename}"

trink · 2022-09-29T23:14:18Z

FYI: This doesn't handle substitution in the keymap arguments.

[keys.normal]
"C-s" = [":sh hx_rcs_write.sh '%val{filename}'", ":w"] # version control the previous and write out the new

helix-term/src/commands/typed.rs

QiBaobin · 2022-11-16T13:20:34Z

"C-s" = [":sh hx_rcs_write.sh '%val{filename}'", ":w"]

@trink , it shall work now, please take anther look, thanks!

QiBaobin · 2022-11-21T02:49:01Z

support %val {filename} %val {dirname} and %sh {cmd} now, we can also nest them like %sh { echo %val {dirname} }, also addressed the 'have to escape space' issue as @the-mikedavis suggested.

book/src/generated/typable-cmd.md

pickfire · 2023-01-25T15:59:30Z

helix-term/src/commands/typed.rs

+    } else if event == PromptEvent::Validate {
+        cx.editor
+            .set_error(format!("no such command: '{}'", parts[0]));
+    }


Shouldn't this return an error too?

pickfire · 2023-01-25T16:07:21Z

helix-term/src/commands/typed.rs

+    for (i, &b) in str.as_bytes().iter().enumerate() {
+        match char::from_u32(b as u32) {


Shouldn't we iterate over char directly instead? I think unicode might crash in this case when we are reading bytes within multiple bytes char.

Also, the count with unicode characters is incorrect now, as_bytes may increment i more times than the actual character length str.len.

the b is a byte so that this won't crash.As we only use ascii chars in our patterns below, I didn't treat them as Unicode char, this shall work until we introduce other Unicode pattern beside %val etc.

If these patterns are coming from user input there's always a chance that they can be multi-byte, or even multi code-point "characters". Because of that you might want the parser to be aware / tolerant of it.

If that is the case then String.chars probably still isn't enough and you'll need something more unicode aware if you're interested in knowing a particular characters index as a human would see it. See this note from the rust std library:

let y = "y̆"; let mut chars = y.chars(); assert_eq!(Some('y'), chars.next()); // not 'y̆'

The typical crate I've used / seen used is https://crates.io/crates/unicode-segmentation . Specifically for this implementation you could use grapheme_indices https://unicode-rs.github.io/unicode-segmentation/unicode_segmentation/trait.UnicodeSegmentation.html#tymethod.grapheme_indices .

The returned index will always be in bounds (the enumerator and len are both byte based). Trying something like the following doesn't cause any issues
sh echo %val{𒀀𒀀𒀀𒀀𒀀𒀀𒀀} // 4 byte char f0928080

I think this should remain as is.

However, downstream I am seeing something off in shellwords (some multi-byte characters are lost in the generated argument list, I am investigating it now but it is unrelated to this PR)
edit: #5738

pickfire · 2023-01-25T16:19:29Z

helix-term/src/commands/typed.rs

+    let reg = Regex::new(r"%(\w+)\s*\{(.*)").unwrap();
+    reg.replace(args, |caps: &regex::Captures| {
+        let remaining = &caps[2];
+        let end = find_first_open_right_braces(remaining);


Should we auto-complete for users? I think %sh{echo hello shouldn't work.

yes, above command won't expand and will be treated literately . The auto-complete is a very nice to have feature, how about we adding it after this PR.

helix-term/src/commands/typed.rs

Co-authored-by: Ivan Tham <pickfire@riseup.net>

trink · 2023-01-29T23:34:23Z

helix-term/src/commands/typed.rs

+            _ => "".into(),
+        };
+        let next = expand_args(editor, remaining.get(end + 1..).unwrap_or(""));
+        format!("{rep} {next}")


The added space between rep and next is a problem when you want to compose a single argument with the substitution. e.g.
:sh echo %val{filename},v
This currently produces two arguments "filename", ",v" instead of the desired "filename,v"

Good point, will fix it once I have time.

trink · 2023-01-29T23:51:56Z

helix-term/src/commands/typed.rs

+    for (i, &b) in str.as_bytes().iter().enumerate() {
+        match char::from_u32(b as u32) {


The returned index will always be in bounds (the enumerator and len are both byte based). Trying something like the following doesn't cause any issues
sh echo %val{𒀀𒀀𒀀𒀀𒀀𒀀𒀀} // 4 byte char f0928080

I think this should remain as is.

However, downstream I am seeing something off in shellwords (some multi-byte characters are lost in the generated argument list, I am investigating it now but it is unrelated to this PR)
edit: #5738

danillos · 2023-02-28T11:07:45Z

Thanks, @QiBaobin I really liked this feature, with that I could build:

F8 = ':open %sh { ~/.dotfiles/support/command.sh switch_to_spec %val{filename} }'
F10 = ':sh ~/.dotfiles/support/command.sh run_current_file_spec %val{filename}'

These scripts that I created are not perfect because I had to hard-code how to get the project dirname.

Could we have something like %val{project_dirname} expand to the absolute folder path used in hx . ?

Edit: Another useful variable is something like %val{line_number}, to use in commands like open it in github.

ksdrar · 2023-04-23T04:48:11Z

Hey guys, I got the nested commands working with %key{body}

You can try the changes in the branch cmd-expansions of my fork.

The output of the tests:

❯ cargo run --release
   Compiling pattern_matching v0.1.0 (/home/jesus/Development/rust/pattern_matching)
    Finished release [optimized] target(s) in 0.54s
     Running `target/release/pattern_matching`
[TEST] %val {filedir} => filedir
[PASSED 821.124µs] filedir == filedir

[TEST] %val {filedir%val {aasdfasdf}} => filediraasdfasdf
[PASSED 738.772µs] filediraasdfasdf == filediraasdfasdf

[TEST] %sh {rm %val{filedir},%val{filename}} => rm filedir,filename
[PASSED 647.805µs] rm filedir,filename == rm filedir,filename

[TEST] %sh {git blame %val{filename}:%val{linenumber}} => git blame filename:linenumber
[PASSED 666.177µs] git blame filename:linenumber == git blame filename:linenumber

[TEST] :sh (%sh {cat %val{filename} | grep rust}) > %val{filedir}/test.txt => :sh (cat filename | grep rust) > filedir/test.txt
[PASSED 660.74µs] :sh (cat filename | grep rust) > filedir/test.txt == :sh (cat filename | grep rust) > filedir/test.txt

[TEST] :sh bash -c (%sh {(%sh {cat %val{filedir}/test.txt}) > %val{filedir}/../%val{filename}_test.txt}) => :sh bash -c ((cat filedir/test.txt) > filedir/../filename_test.txt)
[PASSED 736.72µs] :sh bash -c ((cat filedir/test.txt) > filedir/../filename_test.txt) == :sh bash -c ((cat filedir/test.txt) > filedir/../filename_test.txt)

The bare-bones code:

fn test(to_test: &str, expected: &str) {
    let start = std::time::Instant::now();
    println!("\x1b[33m[TEST]\x1b[0m {to_test} => {expected}");
    let result = find_and_replace(to_test).unwrap_or_default();
    assert_eq!(result, expected);
    println!(
        "\x1b[32m[PASSED {:2?}]\x1b[0m {result} == {expected}\n",
        start.elapsed()
    );
}

fn main() {
    test("%val {filedir}", "filedir");

    test("%val {filedir%val {aasdfasdf}}", "filediraasdfasdf");

    test(
        "%sh {rm %val{filedir},%val{filename}}",
        "rm filedir,filename",
    );

    test(
        "%sh {git blame %val{filename}:%val{linenumber}}",
        "git blame filename:linenumber",
    );

    test(
        ":sh (%sh {cat %val{filename} | grep rust}) > %val{filedir}/test.txt",
        ":sh (cat filename | grep rust) > filedir/test.txt",
    );

    test(
        ":sh bash -c (%sh {(%sh {cat %val{filedir}/test.txt}) > %val{filedir}/../%val{filename}_test.txt})",
        ":sh bash -c ((cat filedir/test.txt) > filedir/../filename_test.txt)",
    );
}

fn find_and_replace(input: &str) -> anyhow::Result<String> {
    let regexp = regex::Regex::new(r"%(\w+)\s*\{([^{}]*(\{[^{}]*\}[^{}]*)*)\}").unwrap();

    replace_all(&regexp, input, |captures| {
        let keyword = captures.get(1).unwrap().as_str();
        let body = captures.get(2).unwrap().as_str();

        match keyword {
            "val" => Ok(String::from(body)),
            "sh" => Ok(String::from(body)),
            _ => anyhow::bail!("Unknown keyword {keyword}"),
        }
    })
}

fn replace_all(
    regex: &regex::Regex,
    text: &str,
    matcher: impl Fn(&regex::Captures) -> anyhow::Result<String>,
) -> anyhow::Result<String> {
    let mut it = regex.captures_iter(text).peekable();

    if it.peek().is_none() {
        return Ok(String::from(text));
    }

    let mut new = String::with_capacity(text.len());
    let mut last_match = 0;

    for cap in it {
        let m = cap.get(0).unwrap();
        new.push_str(&text[last_match..m.start()]);

        let replace = matcher(&cap)?;

        new.push_str(&replace);

        last_match = m.end();
    }

    new.push_str(&text[last_match..]);

    replace_all(regex, &new, matcher)
}

danillos · 2023-04-24T22:37:15Z

I’ve tested your branch, @ksdrar, and it worked perfectly. In my opinion, you can submit a Pull Request to replace the current one. I believe @QiBaobin will close this current PR once yours is opened.

Nice work.

danillos · 2023-04-24T23:21:16Z

@ksdrar I found an issue. I was migrating all keybindings that I did using my version to your version and this one didn't work.

# before
s = ':open %sh { ~/.dotfiles/helix/command.rb switch_to_spec %val{projectdir} %val{relative_filename}}'

# after
s = ':open %sh {~/.dotfiles/helix/command.rb switch_to_spec %sh{pwd} %val{filename}}'

It is not expanding %sh{pwd} and %val{filename}, If I understood, it first expands %sh

danillos · 2023-04-25T00:35:31Z

I think it should expand the most inner expressions first, and move to the next one only when there is nothing left to expand. For example:

filename = test.rs
line_number = 123

Command:
%sh { add_suffix.sh %sh { add_line_number.sh %sh { add_prefix.sh %val{filename} } } }

Expansion Steps:

0. %sh { add_suffix.sh %sh { add_line_number.sh %sh { add_prefix.sh %val{filename} } } }
1. %sh { add_suffix.sh %sh { add_line_number.sh %sh { add_prefix.sh test.rs } } }
2. %sh { add_suffix.sh %sh { add_line_number.sh prefix_test.rs } }
3. %sh { add_suffix.sh prefix_test.rs:%val{line_number} }
4. %sh { add_suffix.sh prefix_test.rs:123 }
5. prefix_test_suffix.rs:123

Scripts:

add_prefix.sh: adds `prefix_{string}` to a string
add_suffix.sh: adds `{string}_suffix` to a string
add_line_number.sh: adds `{string}:%val{line_number}` to a string

ksdrar · 2023-04-25T01:15:59Z

@danillos Can you check if it's working now with the last commit?

danillos · 2023-04-25T01:22:50Z

@danillos Can you check if it's working now with the last commit?

Yes, this example is now working.

s = ':open %sh {~/.dotfiles/helix/command.rb switch_to_spec %sh{pwd} %val{filename}}'

danillos · 2023-05-06T11:46:19Z

@ksdrar could you open your changes as a new PR for replace this one? I could do that too.

Based on: helix-editor#3393 --- Squashed commit of the following: commit 4e16272eb27fde410022785ffcdb6828ac61aee9 Merge: 07b109d 5ae30f1 Author: Jesús R <roldanr.jesus@gmail.com> Date: Sat May 6 07:56:28 2023 -0600 Merge branch 'master' into cmd-expansions commit 07b109d Author: Jesús R <roldanr.jesus@gmail.com> Date: Mon Apr 24 19:14:58 2023 -0600 fix: Not parsing sh body commit 8bf040f Author: Jesús R <roldanr.jesus@gmail.com> Date: Sat Apr 22 22:46:17 2023 -0600 Use %key{body} commit f6fea44 Merge: ffb40de b7c62e2 Author: Jesús R <roldanr.jesus@gmail.com> Date: Sat Apr 22 17:34:07 2023 -0600 Merge branch 'master' into cmd-expansions commit ffb40de Merge: 2be5d34 b9b4ed5 Author: Jesús R <roldanr.jesus@gmail.com> Date: Tue Apr 11 15:48:15 2023 -0600 Merge branch 'master' into cmd-expansions commit 2be5d34 Author: Jesús R <roldanr.jesus@gmail.com> Date: Tue Apr 11 15:46:58 2023 -0600 Use #{} for variables and #key [] for commands commit 7e7c0dc Merge: 22d17f9 531b745 Author: Bob <QiBaobin@users.noreply.github.com> Date: Wed Apr 5 09:11:13 2023 +0800 Merge branch 'master' into cmd-expansions commit 22d17f9 Author: Bob <QiBaobin@users.noreply.github.com> Date: Thu Jan 26 09:43:13 2023 +0800 Update helix-term/src/commands/typed.rs Co-authored-by: Ivan Tham <pickfire@riseup.net> commit 85d38a7 Author: Bob Qi <baobin.qi@gmail.com> Date: Tue Jan 24 14:49:16 2023 +0800 remove command group function commit 784380f Merge: 6f6cb3c 64ec025 Author: Bob Qi <baobin.qi@gmail.com> Date: Tue Jan 24 14:42:31 2023 +0800 Merge remote-tracking branch 'origin/master' commit 6f6cb3c Author: Bob Qi <baobin.qi@gmail.com> Date: Mon Nov 21 10:39:52 2022 +0800 support commmand expansion

AlexanderBrevig mentioned this pull request Sep 3, 2022

Ability to use $file and $pwd in shell commands and formatter args #3642

Closed

3 tasks

kirawi added A-helix-term Area: Helix term improvements S-waiting-on-review Status: Awaiting review from a maintainer. labels Sep 13, 2022

the-mikedavis reviewed Oct 15, 2022

View reviewed changes

helix-term/src/commands/typed.rs Outdated Show resolved Hide resolved

QiBaobin mentioned this pull request Nov 16, 2022

Close buffer and delete underlying file #4592

Open

QiBaobin requested a review from the-mikedavis November 16, 2022 13:47

support commmand expansion

6f6cb3c

This was referenced Nov 21, 2022

add cmds to support command chain #4773

Closed

use % as file name register #3008

Closed

archseer mentioned this pull request Jan 19, 2023

Add % current file register #5577

Closed

archseer added this to the next milestone Jan 19, 2023

the-mikedavis reviewed Jan 21, 2023

View reviewed changes

book/src/generated/typable-cmd.md Outdated Show resolved Hide resolved

the-mikedavis mentioned this pull request Jan 21, 2023

feat(format): current file in formatter args #5626

Open

QiBaobin added 2 commits January 24, 2023 14:42

Merge remote-tracking branch 'origin/master'

784380f

remove command group function

85d38a7

pickfire reviewed Jan 25, 2023

View reviewed changes

helix-term/src/commands/typed.rs Outdated Show resolved Hide resolved

Update helix-term/src/commands/typed.rs

22d17f9

Co-authored-by: Ivan Tham <pickfire@riseup.net>

trink suggested changes Jan 29, 2023

View reviewed changes

This was referenced Feb 22, 2023

Keymap presentation refinements and underlying restructuring #5635

Closed

Unify and clarify config loading #5636

Closed

pascalkuthe added the S-inactive Status: Inactive and waiting on the author. This is often applied to closed PRs. label Apr 14, 2023

the-mikedavis mentioned this pull request Apr 24, 2023

Custom typable commands #4423

Open

QiBaobin closed this May 5, 2023

ksdrar mentioned this pull request May 6, 2023

Add command expansions %key{body} #6979

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

command expansions #3393

command expansions #3393

QiBaobin commented Aug 11, 2022

QiBaobin commented Aug 11, 2022 •

edited

Loading

sbromberger commented Aug 12, 2022

QiBaobin commented Aug 12, 2022

trink commented Sep 29, 2022

QiBaobin commented Nov 16, 2022 •

edited

Loading

QiBaobin commented Nov 21, 2022

pickfire Jan 25, 2023

pickfire Jan 25, 2023 •

edited

Loading

QiBaobin Jan 26, 2023

g-re-g Jan 27, 2023 •

edited

Loading

trink Jan 29, 2023 •

edited

Loading

pickfire Jan 25, 2023

QiBaobin Jan 26, 2023

trink Jan 29, 2023

QiBaobin Jan 30, 2023

trink Jan 29, 2023 •

edited

Loading

danillos commented Feb 28, 2023 •

edited

Loading

ksdrar commented Apr 23, 2023

danillos commented Apr 24, 2023 •

edited

Loading

danillos commented Apr 24, 2023 •

edited

Loading

danillos commented Apr 25, 2023 •

edited

Loading

ksdrar commented Apr 25, 2023

danillos commented Apr 25, 2023 •

edited

Loading

danillos commented May 6, 2023

		for (i, &b) in str.as_bytes().iter().enumerate() {
		match char::from_u32(b as u32) {

command expansions #3393

command expansions #3393

Conversation

QiBaobin commented Aug 11, 2022

QiBaobin commented Aug 11, 2022 • edited Loading

sbromberger commented Aug 12, 2022

QiBaobin commented Aug 12, 2022

trink commented Sep 29, 2022

QiBaobin commented Nov 16, 2022 • edited Loading

QiBaobin commented Nov 21, 2022

pickfire Jan 25, 2023

Choose a reason for hiding this comment

pickfire Jan 25, 2023 • edited Loading

Choose a reason for hiding this comment

QiBaobin Jan 26, 2023

Choose a reason for hiding this comment

g-re-g Jan 27, 2023 • edited Loading

Choose a reason for hiding this comment

trink Jan 29, 2023 • edited Loading

Choose a reason for hiding this comment

pickfire Jan 25, 2023

Choose a reason for hiding this comment

QiBaobin Jan 26, 2023

Choose a reason for hiding this comment

trink Jan 29, 2023

Choose a reason for hiding this comment

QiBaobin Jan 30, 2023

Choose a reason for hiding this comment

trink Jan 29, 2023 • edited Loading

Choose a reason for hiding this comment

danillos commented Feb 28, 2023 • edited Loading

ksdrar commented Apr 23, 2023

danillos commented Apr 24, 2023 • edited Loading

danillos commented Apr 24, 2023 • edited Loading

danillos commented Apr 25, 2023 • edited Loading

ksdrar commented Apr 25, 2023

danillos commented Apr 25, 2023 • edited Loading

danillos commented May 6, 2023

QiBaobin commented Aug 11, 2022 •

edited

Loading

QiBaobin commented Nov 16, 2022 •

edited

Loading

pickfire Jan 25, 2023 •

edited

Loading

g-re-g Jan 27, 2023 •

edited

Loading

trink Jan 29, 2023 •

edited

Loading

trink Jan 29, 2023 •

edited

Loading

danillos commented Feb 28, 2023 •

edited

Loading

danillos commented Apr 24, 2023 •

edited

Loading

danillos commented Apr 24, 2023 •

edited

Loading

danillos commented Apr 25, 2023 •

edited

Loading

danillos commented Apr 25, 2023 •

edited

Loading